The present invention relates generally to event recording systems, and more particularly to systems for debriefing event participants.
It is well known throughout the airline industry that flight crew error is a contributing factor in a large number of aircraft accidents. However, many of the errors made by multi-person flight crews have very little to do with the technical aspects of flying an aircraft. Instead, a large number of these errors are related to poor group decision making, ineffective communication, inadequate leadership, and poor task management (collectively referred to as xe2x80x9chuman factorsxe2x80x9d errors) among the flight crew.
Flight simulators were developed in the late 1960""s and early 1970""s to increase flight crew proficiency in the technical aspects of flying. Flight simulators are computer controlled devices which simulate actual flying conditions of an aircraft and train flight crews to deal with a variety of situations before they actually encounter them in a real aircraft. Early on, the airline industry recognized the importance of increasing flight crew proficiency with respect to eliminating human factors errors, and began combining both technical and human factors training during flight simulator exercises. For example, a high-stress environment that requires clear, unequivocal communication between all of the crew members is the loss of an engine on take off. Although often regarded as a technical maneuver, it also requires accurate information transfer between the flight crew, the control tower, and the cabin crew.
Increasing the proficiency of flight crews with respect to human factors knowledge and skills has come to be known as xe2x80x9ccockpit resource managementxe2x80x9d (CRM) training. CRM training typically involves videotaping a flight simulator training exercise for later playback and review by the flight crew during debriefing. Simulators equipped with video cameras can help flight crews sharpen their communication skills and other human factors skills in various emergency situations. Unfortunately, conventional video cassette recorder (VCR) technology presently used to record flight simulator exercises is often inadequate during debriefing. Rewinding and forwarding is often required in order to locate a particular portion of an event recorded on videotape, and to review a portion of the videotape multiple times. Some degree of time lag is typically involved because it is rather difficult to quickly jump back and forth between different portions of the videotape. Another disadvantage of using conventional VCR technology for capturing a flight simulator training exercise for debriefing purposes is that it is somewhat difficult to simultaneously capture with video cameras, flight data as represented by the various indicators and dials in the simulator, and the actions of the flight crew.
U.S. Pat. No. 5,467,274 describes a method for debriefing aircraft operations. As described, aircraft operations are recorded by both video recorders and audio recorders and are stored by an airborne videotape recorder (AVTR). The AVTR also records selected flight data as well as an associated time parameter. During debriefing operations, the various signals that have been recorded, such as video signals, audio signals and other signals relating to flight parameters, are synchronized and displayed upon a three dimensional display.
Published PCT application No. WO 95/25313 (hereinafter the ""313 application) is not directed to a debriefing system or method but, instead, describes a method and apparatus for synchronizing and scheduling multiple datastreams, such as required in a variety of multi-media applications including computer-integrated telephony. The apparatus of ""313 application includes a selector that tracks the status of each of the datastreams. In addition, a start pointer and an end pointer are associated with each datastream. Once the system time equals the start time of a respective datastream, the selector updates its status to indicate if the start time has been reached. Each datastream also generally includes a trigger condition. Once the start time has been reached, the selector therefore determines that the trigger condition has been met and, if so, forwards the respective datastream to a playout device. In order to synchronize multiple datastreams, the trigger conditions of the datastreams may require that the start times of two or more of the datastreams have been reached such that each of the datastreams is forwarded to the playout device at the same time.
It is therefore an object of the present invention to provide improved debriefing systems and methods.
It is another object of the present invention to provide improved debriefing systems and methods that overcome the limitations and drawbacks of conventional VCR technology.
These and other objects are accomplished, according to the present invention, by event debriefing systems and methods for recording and replaying various types of information generated during an event, both simulated and real. Facilities are provided for recording a plurality of datastreams related to an event, each datastream including chronological information related to at least one aspect of the event. Recorded datastreams include video recordings of the actions of event participants, and audio recording of audible communications of event participants. Also included are instrumentation datastream from various event-monitoring instruments, including those necessary to create aircraft flight animation.
Facilities are provided whereupon, any point in time during an event can be xe2x80x9cindicatedxe2x80x9d with a time indication mark, created either by an observer or automatically by the debriefing system. An observer of the event may create time indication marks that facilitate recalling, during debriefing, what transpired at any particular point in time during the event. The observer is presented with, preferably via a computer display, menus of categories and sub-categories of pre-defined time indication marks that can be selected and associated with the vent. Similarly, the debriefing system may be configured to automatically create time indication marks when certain parameters are met during the event.
Time indication marks may be defined for virtually any aspect of an event including, but not limited to, categories related to the actions of participants in the event (CRM for example), and categories related to technical aspects of the event. By associating a time during the event with a particular category of time indication marks, selective replay of the various recorded datastreams is facilitated during debriefing. For example, if communications between event participants is good (or bad) at a particular point in time during an event, a pre-defined time indication mark may be selected from a menu and associated with each of the recorded datastreams. During debriefing, the event participants can view the datastreams in time synchronism beginning at or near this time indication mark. Time indication marks that are not pre-defined may also be created and associated with the recorded datastreams. For example, if something were to occur during an event that would be worth reviewing during debriefing, yet does not fall within a particular pre-defined category, an observer can create a time indication mark anyway.
The recorded datastreams are retrieved and presented in time synchronism. In particular, the recorded datastreams can be retrieved and presented beginning at marked time indications and continued chronologically thereafter in time synchronism. During debriefing, a user is presented with various computer screens from which particular time indication marks can be selected. For example, a list of all marks created during an event may be listed. Alternatively, various data from an event, with associated time indication marks, may be presented for selection to a user. By selecting a particular time indication mark, the various recorded datastreams are replayed in time synchronism beginning at or near the time associated with the selected time indication mark.
A display having a plurality of zones is used to present video and instrumentation datastreams in time synchronism. Event-monitoring instrumentation and other data is displayed in dynamic animation. An audio datastream is performed in time synchronism with the displayed video and instrumentation datastreams. Consequently, the disadvantages of conventional VCR technology are overcome by the present invention. Any portion of a recorded event can be reviewed at the press of a button, with virtually no time lag.
The present invention may be used in conjunction with aircraft flight simulator debriefing systems and methods. Data, from both the flight simulator and the video camera on the simulator flight deck, is directed to a computer. A computer screen is configured to display a variety of pre-defined categories of time indication marks. A training facilitator, referred to as a xe2x80x9ccheck pilotxe2x80x9d, observes crew interactions during the simulator exercise and selects time indication marks from the display. The time indication marks may be associated with various aspects of the simulated flight, including technical and CRM aspects. For example, if the check pilot observes flight crew behavior during a simulated event that would be desirable to review during debriefing, he or she can select a pre-defined time indication mark, or create a mark that is not pre-defined, and associate the mark with the recorded datastreams.
Software associated with the debriefing computer automatically reconstructs the simulator flight, links it to the video and audio signal from the flight deck and displays the digitized video, dynamic animation and selected instrumentation on the monitor located in a debriefing room. During debriefing following a simulator exercise, a check pilot is able to go to any event during the flight and show the crew exactly what transpired on the flight deck, complete with animated instrumentation and multiple views of the aircraft and flight path, all on a single monitor. Voice recognition software may also be included wherein the conversations of the crew can be searched for selected words and phrases during the simulator exercise.
The present invention is advantageous because it substantially increases training efficiency and effectiveness by allowing the crew to xe2x80x9crelivexe2x80x9d their flight experience in a single integrated display of their technical and human factor performance. Another advantage of the debriefing system, according to the present invention, is that a flight crew member can debrief himself or herself after a training exercise. This removes the subjectivity of the check pilot and puts him or her in the position of having to simply manage the interaction between the flight crew and the simulator system. As an additional benefit, the present invention will result in increased standardization of check pilot briefings, debriefings, and evaluations.
Prior to the present invention, marks on videotape lacked identifying information. The check pilot knew that a mark existed, but there was not any descriptive information associated with the mark. Time indication marks, according to the present invention, facilitate the recall of events that transpired in the simulated exercise, particularly at critical junctures, when the exercise is replayed during the debriefing session. Because there is a tremendous amount of stimuli occurring (for example, the flight instruments move dynamically, the visual scenes outside of the cockpit move, and various audio communications take place, including bells, horns, communications between the flight crew and with air traffic control) during a simulated flight exercise, it is difficult for the flight crew and the check pilot to recall all of the events and data that occurred during a session. The cognitive processes of the flight crew and the check pilot cannot assimilate all the stimuli in order to effectively train motor skills and cognitive skills.
As described above, the invention implements systems and methods for: (1) recording, marking, and presenting multiple datastreams in time synchronism; (2) recording, categorized marking, and presenting multiple datastreams in time synchronism; and (3) recording, automatically marking, and presenting multiple datastreams in time synchronism. Each of these aspects may be used individually or in combination, to improve training efficiency and effectiveness during debriefing of an event.
The present invention is not limited to the recording and debriefing of simulated events. Many xe2x80x9crealxe2x80x9d events can be recorded and the actual event participants debriefed using the present invention. For example, the systems and methods of the present invention can be utilized in the control room of a nuclear power plant, the operating room of a hospital, the control center of the National Aeronautics and Space Administration (NASA) during a space mission, the assembly line of a manufacturing plant, and many other locations involving both technical aspects of an event and interaction among participants in the event. The present invention is a powerful learning tool with many potential applications.